{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c6a2bd60",
   "metadata": {},
   "source": [
    "# FTTransformer回归任务"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "382eb806",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "本范例演示使用 torchkeras.tabular 中的 FTTransformer 对加州房价数据集 进行回归建模。\n",
    "\n",
    "公众号**算法美食屋**后台回复关键词：torchkeras，获取本文notebook源码和所用california_housing数据集下载链接。\n",
    "\n",
    "\n",
    "California_housing数据集使用房屋的以下一些特征:\n",
    "\n",
    "* 房屋所在街区中位数收入(MedInc)\n",
    "  \n",
    "* 房屋年龄(HouseAge)\n",
    "\n",
    "* 房屋的房间数量(AveRooms)\n",
    "\n",
    "* 房屋平均每个房间的床数量(AveBedrms)\n",
    "\n",
    "* 房屋所在地区的人口(Population)\n",
    "\n",
    "* 房屋所在街区平均家庭人数(AveOccup)\n",
    "\n",
    "* 房屋位置经度(Longitude)\n",
    "\n",
    "* 房屋位置纬度(Latitude)\n",
    "\n",
    "\n",
    "来预测房屋的中位数每平米单价(MedHouseVal)。\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "79c9440c-4e00-4d27-93d6-3f73c40ae149",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import sys\n",
    "sys.path.append('..')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "81c4a29a",
   "metadata": {},
   "source": [
    "## 一，准备数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "3d9dfe1d-5f6e-4e14-ae76-3841ac15fb1b",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dfdata.shape =  (20640, 9)\n",
      "target_col =  MedHouseVal\n",
      "cat_cols =  []\n",
      "num_cols =  ['MedInc', 'HouseAge', 'AveRooms', '...']\n",
      "len(dftrain) =  14860\n",
      "len(dfval) =  1652\n",
      "len(dftest) =  4128\n"
     ]
    }
   ],
   "source": [
    "import numpy as np \n",
    "import pandas as pd \n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "file_path = \"california_housing.parquet\"\n",
    "dfdata = pd.read_parquet(file_path)\n",
    "\n",
    "num_cols = ['MedInc', 'HouseAge', 'AveRooms', 'AveBedrms', 'Population', 'AveOccup',\n",
    "       'Latitude', 'Longitude']\n",
    "cat_cols = []\n",
    "target_col = 'MedHouseVal'\n",
    "\n",
    "\n",
    "print(\"dfdata.shape = \",dfdata.shape)\n",
    "print(\"target_col = \", target_col)\n",
    "print('cat_cols = ', cat_cols)  \n",
    "print('num_cols = ', num_cols[:3]+['...'])\n",
    "\n",
    "dftmp, dftest_raw = train_test_split(dfdata, random_state=42, test_size=0.2)\n",
    "dftrain_raw, dfval_raw = train_test_split(dftmp, random_state=42, test_size=0.1)\n",
    "\n",
    "print(\"len(dftrain) = \",len(dftrain_raw))\n",
    "print(\"len(dfval) = \",len(dfval_raw))\n",
    "print(\"len(dftest) = \",len(dftest_raw))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "169703c2",
   "metadata": {},
   "outputs": [],
   "source": [
    "from torchkeras.tabular import TabularPreprocessor\n",
    "from sklearn.preprocessing import OrdinalEncoder\n",
    "\n",
    "#特征工程\n",
    "pipe = TabularPreprocessor(cat_features = cat_cols, \n",
    "                           embedding_features=cat_cols)\n",
    "\n",
    "dftrain = pipe.fit_transform(dftrain_raw.drop(target_col,axis=1))\n",
    "dftrain[target_col] = dftrain_raw[target_col]\n",
    "\n",
    "dfval = pipe.transform(dfval_raw.drop(target_col,axis=1))\n",
    "dfval[target_col] = dfval_raw[target_col]\n",
    "\n",
    "dftest = pipe.transform(dftest_raw.drop(target_col,axis=1))\n",
    "dftest[target_col] = dftest_raw[target_col]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "7768b968",
   "metadata": {},
   "outputs": [],
   "source": [
    "from torchkeras.tabular import TabularDataset\n",
    "from torch.utils.data import Dataset,DataLoader \n",
    "\n",
    "def get_dataset(dfdata):\n",
    "    return TabularDataset(\n",
    "                data = dfdata,\n",
    "                task = 'regression', #regression, binary, multiclass\n",
    "                target = [target_col],\n",
    "                continuous_cols = pipe.get_numeric_features(),\n",
    "                categorical_cols = pipe.get_embedding_features()\n",
    "        )\n",
    "\n",
    "def get_dataloader(ds,batch_size=1024,num_workers=0,shuffle=False):\n",
    "    dl = DataLoader(\n",
    "            ds,\n",
    "            batch_size=batch_size,\n",
    "            shuffle=shuffle,\n",
    "            num_workers=num_workers,\n",
    "            pin_memory=False,\n",
    "        )\n",
    "    return dl \n",
    "    \n",
    "ds_train = get_dataset(dftrain)\n",
    "ds_val = get_dataset(dfval)\n",
    "ds_test = get_dataset(dftest)\n",
    "\n",
    "dl_train = get_dataloader(ds_train,shuffle=True)\n",
    "dl_val = get_dataloader(ds_val,shuffle=False)\n",
    "dl_test = get_dataloader(ds_test,shuffle=False)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "ec009434",
   "metadata": {},
   "outputs": [],
   "source": [
    "for batch in dl_train:\n",
    "    break\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "4c65b12d-aa68-4103-bbae-8ecc0fd52e83",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([ 5438,  9242,  6999,  ..., 13178,  7561, 14200])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "batch['id']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb2f0170",
   "metadata": {},
   "source": [
    "## 二，定义模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "64901291-6ff3-4cd8-90e4-9bd36e91e401",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from torchkeras.tabular.models import FTTransformerConfig,FTTransformerModel\n",
    "\n",
    "model_config = FTTransformerConfig(\n",
    "    task=\"regression\",  #regression, binary, multiclass\n",
    "    num_attn_blocks=4\n",
    ")\n",
    "\n",
    "config = model_config.merge_dataset_config(ds_train)\n",
    "net = FTTransformerModel(config = config)\n",
    "\n",
    "# 考虑到房屋价格总是正的，将模型的Head重置为PoissonHead (激活函数为torch.exp)\n",
    "# 将模型的 Loss重置为 PoissonLoss \n",
    "\"\"\"\n",
    "class PoissonHead(nn.Module):\n",
    "    def __init__(self,in_features,out_features):\n",
    "        super().__init__()\n",
    "        self.linear = nn.Linear(in_features = in_features,out_features=out_features)    \n",
    "    def forward(self,x):\n",
    "        return torch.exp(self.linear(x))\n",
    "    \n",
    "class PoissonLoss(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(PoissonLoss, self).__init__()\n",
    "        self.loss_fn = nn.PoissonNLLLoss(log_input=False,full=True)\n",
    "\n",
    "    def forward(self, inputs, targets):\n",
    "        loss = self.loss_fn(inputs,targets)\n",
    "        return loss.mean()\n",
    "    \n",
    "print(net.backbone.output_dim)\n",
    "print(net.hparams.output_dim)\n",
    "\n",
    "net._head  = PoissonHead(in_features=net.backbone.output_dim,\n",
    "                         out_features=net.hparams.output_dim)\n",
    "net.loss = PoissonLoss()\n",
    "\"\"\"\n",
    "\n",
    "net.loss = nn.HuberLoss()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "6947e24f-2468-4c9c-a0a1-28d18225d409",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "32\n",
      "1\n"
     ]
    }
   ],
   "source": [
    "#初始化参数\n",
    "net.reset_weights()\n",
    "net.data_aware_initialization(dl_train)\n",
    "\n",
    "print(net.backbone.output_dim)\n",
    "print(net.hparams.output_dim)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "7e312c76",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(1.4124, grad_fn=<AddBackward0>)\n"
     ]
    }
   ],
   "source": [
    "output = net.forward(batch)\n",
    "loss = net.compute_loss(output,batch['target'])\n",
    "print(loss)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c24560be-5197-46f8-90e8-49285a0e5ab0",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "22001eaf",
   "metadata": {},
   "source": [
    "## 三，训练模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "14bb01e8",
   "metadata": {},
   "outputs": [],
   "source": [
    "from torchkeras import KerasModel \n",
    "from torchkeras.tabular import StepRunner \n",
    "KerasModel.StepRunner = StepRunner \n",
    "\n",
    "\n",
    "import torch \n",
    "from torchmetrics import MeanAbsoluteError \n",
    "\n",
    "keras_model = KerasModel(net,\n",
    "                   loss_fn=None,\n",
    "                   optimizer = torch.optim.AdamW(net.parameters(),lr = 3e-4),\n",
    "                   metrics_dict = {\"mae\":MeanAbsoluteError()}\n",
    "                   )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "08f4ad31",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;31m<<<<<< 🚀 mps is used >>>>>>\u001b[0m\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhgAAAGJCAYAAADIVkprAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABn/0lEQVR4nO3deVhUZfsH8O+ZAQYQAdkXUVRwXzAXcsslijT3zLXcSq20RPJ909RMM6lfaZhplpmmmUtGZlmWkpb7mr6uKAqCBAjKIijbzPP7Y5yRYQYYYGBYvp/rOpfOmeecc89hOTfPKgkhBIiIiIhMSGbuAIiIiKj2YYJBREREJscEg4iIiEyOCQYRERGZHBMMIiIiMjkmGERERGRyTDCIiIjI5JhgEBERkckxwSAiIiKTY4JBJvXuu+9CkiSkpqaaO5QqExsbC0mSsGHDBnOHQmayfft2ODk5ISsry9yhVJkNGzZAkiScOnXK3KFozZkzB4GBgeYOgx5igkG1wtKlS7Fz505zh1FnJSQkYOTIkXB0dIS9vT2GDBmCGzduGHXs0qVL8fjjj8PV1RXW1tbw9/dHSEgIUlJSdMr9+++/eOGFF9CiRQvUr18fjo6O6Nq1K7755huUtuLBU089BUmSMGPGDL33Pv/8czz//PNo1KgRJEnCxIkTjf7cAKBUKrFw4UK8/vrrsLOzK7V8Xl4ecnJyynSN2ur06dMYOHAgPDw8YGdnh/bt2+PTTz+FUqnUKZeVlYWQkBA0bNgQCoUCrVq1wueff653vpCQEJw7dw67du2qqo9AJbAwdwBEprB06VKMGDECQ4cONXcodU5WVhb69u2LjIwMvP3227C0tMQnn3yC3r174+zZs3B2di7x+NOnTyMgIACjR49G/fr1cfnyZaxduxa7d+/G2bNnUa9ePQBAamoqbt26hREjRqBRo0bIz8/H3r17MXHiRERFRWHp0qUGzx8REYGjR48We/0PP/wQ9+7dQ9euXZGYmFjmz//zzz8jKioKU6dOLbZMVFQUPvnkE+zevRu3bt0CALi5ueHZZ5/FjBkz8Nhjj5X5ujXd6dOn0b17d/j7++Ott96Cra0tfvvtN8ycORPXr1/HihUrAKgTuODgYJw6dQrTp0+Hv78/fv/9d7z22mtIS0vD22+/rT2nh4cHhgwZgo8//hiDBw8210cjDUFkQgsXLhQAREpKSpVet169emLChAlVek2NmJgYAUCsX7/eLNc3tw8//FAAECdOnNDuu3z5spDL5WLu3LnlOueOHTsEALFly5ZSyw4cOFDUq1dPFBQU6L334MED4evrKxYvXiwAiOnTp+uViY2NFSqVSghRvu+jwYMHi549exb7/pIlS4SFhYUICAgQS5YsETt37hQ//fSTWL58uejZs6eQy+Xi7bffLtM1q4P169cLAOLkyZPlOn7KlCnCyspK3LlzR2f/E088Iezt7bWvt2/fLgCIdevW6ZR77rnnhLW1tUhOTtbZv2PHDiFJkrh+/Xq54iLTYRMJVYrU1FSMHDkS9vb2cHZ2xsyZMw1WC3/77bfo1KkTbGxs4OTkhNGjRyM+Pl6nzLVr1/Dcc8/Bw8MD1tbWaNiwIUaPHo2MjAwAgCRJyM7OxjfffANJkkqs5k5OToaFhQUWLVqk915UVBQkScJnn30GALh79y5mz56Ndu3awc7ODvb29ujfvz/OnTtXwbsDHDhwAJIkYfv27Vi0aBG8vb1Rv359jBgxAhkZGcjNzUVISAjc3NxgZ2eHSZMmITc3V+cc69evR79+/eDm5gaFQoHWrVsbrDYGgN9++w29evVCvXr1UL9+fTz77LO4ePFihT8HAOzYsQNdunRBly5dtPtatmyJJ598Etu3by/XOX19fQEA6enpRpW9f/8+8vLy9N77v//7P6hUKsyePbvY4xs3bgxJksoVZ05ODvbs2YOgoCCD78+fPx9LlizBunXr8M8//2DevHkYMmQIBg8ejFmzZuHgwYPYtWsXPv/8c8ydO1fv+PT0dISEhMDHxwcKhQJ+fn748MMPoVKptGU0fYA+/vhjfPLJJ2jcuDFsbGzQu3dvXLhwQe+cf/75p/Z7wdHREUOGDMHly5f1yiUkJOCll16Cl5cXFAoFmjRpgldffVXvPufm5iI0NBSurq6oV68ehg0bpte8ZUhmZiasra3h6Oios9/T0xM2Njba1wcPHgQAjB49Wqfc6NGjkZOTg59++klnv+ZrUXQ/mYG5MxyqXTQ1GO3atRODBg0Sn332mXjhhRcEAPHiiy/qlF2yZImQJEmMGjVKrF69WixatEi4uLgIX19fkZaWJoQQIjc3VzRp0kR4eXmJJUuWiK+++kosWrRIdOnSRcTGxgohhNi0aZNQKBSiV69eYtOmTWLTpk3iyJEjxcbYr18/0bp1a739ixYtEnK5XCQlJQkhhDh58qRo1qyZmDNnjvjiiy/E4sWLhbe3t3BwcBAJCQna48pTg7F//34BQAQEBIhu3bqJTz/9VLzxxhtCkiQxevRoMXbsWNG/f3+xatUq8eKLLwoAYtGiRTrn6NKli5g4caL45JNPxMqVK8XTTz8tAIjPPvtMp9zGjRuFJEnimWeeEStXrhQffvih8PX1FY6OjiImJkZbLicnR6SkpBi1aSiVSqFQKMSrr76q9xnnz58vAIjMzMxS74dKpRIpKSkiMTFR/P3336J79+5CLpeLy5cv65W9f/++SElJETExMWLDhg2iXr16onv37nrlbt68KWxsbLS1ICimBqOwstZgHDp0SAAQu3bt0nvvr7/+EnK5XOzbt09n/71797Q1Jnfv3hV5eXnizJkzwsbGRhw9elRbLjs7W7Rv3144OzuLt99+W6xZs0aMHz9eSJIkZs6cqS2n+f5r166d8PX1FR9++KFYtGiRcHJyEq6urtrvZyGE2Lt3r7CwsBDNmzcX//d//6f9mWvQoIHO90JCQoLw8vIStra2IiQkRKxZs0YsWLBAtGrVSvuzqanB6Nixo+jXr59YuXKlePPNN4VcLhcjR44s9d59/vnnAoB4+eWXxaVLl0RsbKz4/PPPhaWlpQgPD9eWmzp1qpDL5SI/P1/n+N27dwsAYtq0aXrn9vPzE88991ypMVDlYoJBJqVJMAYPHqyz/7XXXhMAxLlz54QQ6mppuVwu3n//fZ1y58+fFxYWFtr9//zzjwAgvv/++xKvW5YHwxdffCEAiPPnz+vsb926tejXr5/2dU5OjlAqlTplYmJihEKhEIsXL9bZV94Eo23btiIvL0+7f8yYMUKSJNG/f3+d8t26dRONGzfW2Xf//n298wYHB4umTZtqX9+7d084OjqKKVOm6JRLSkoSDg4OOvs1DwxjNo2UlBQBQOd+aKxatUoAEFeuXCn1fiQmJuqcv2HDhmLbtm0Gy4aFhemUffLJJ0VcXJxeuREjRugkHpWRYHz11VcGv5eEEKJPnz4iJCRE+/rIkSPC399fABCurq5i48aNonHjxmL//v1CCCFmzZolxo4dqy3/3nvviXr16omrV6/qnHfOnDlCLpdrP7Pm+8/GxkbcunVLW+748eMCgJg1a5Z2X0BAgHBzc9Npljh37pyQyWRi/Pjx2n3jx48XMpnMYPOHJjnSfL8EBQVp92k+h1wuF+np6SXeu4KCAjFjxgxhaWmp/VrK5XLx+eef65RbtmyZACAOHjyodx8AiIEDB+qd++mnnxatWrUq8fpU+dhEQpVi+vTpOq9ff/11AMCvv/4KQN3xTqVSYeTIkUhNTdVuHh4e8Pf3x/79+wEADg4OAIDff/8d9+/fN0lsw4cPh4WFBbZt26bdd+HCBVy6dAmjRo3S7lMoFJDJ1D8iSqUSd+7cgZ2dHVq0aIEzZ86YJJbx48fD0tJS+zowMBBCCEyePFmnXGBgIOLj41FQUKDdV7gaOSMjA6mpqejduzdu3LihbT7au3cv0tPTMWbMGJ37LJfLERgYqL3PABAcHIy9e/catWk8ePBAe6+Ksra21ilTEicnJ+zduxc///wzFi9eDBcXl2KHfI4ZMwZ79+7Fd999h7Fjxxq8xv79+/HDDz8gPDy81GtXxJ07dwAADRo00NmfkpKCv//+W/t9n52drW3m2759O95//30sWLAASUlJ2mOGDh2KAwcOaF9///336NWrFxo0aKDztQsKCoJSqcTff/+tc82hQ4fC29tb+7pr164IDAzU/swlJibi7NmzmDhxIpycnLTl2rdvj6eeekpbTqVSYefOnRg0aBA6d+6s95mLNidNnTpVZ1+vXr2gVCpx8+bNEu+dXC5Hs2bNEBwcjG+++Qbbtm3DoEGD8Prrr+uMCBs7diwcHBwwefJk7N27F7Gxsfjyyy+xevVqAIa/vzT3jMyLo0ioUvj7++u8btasGWQyGWJjYwGo+1UIIfTKaWgeuk2aNEFoaCiWL1+OzZs3o1evXhg8eDBeeOEFbfJRVi4uLtr+Ae+99x4AYNu2bbCwsMDw4cO15VQqFVasWIHVq1cjJiZGZ+hcaSMjjNWoUSOd15rP5OPjo7dfpVIhIyNDe+3Dhw9j4cKFOHr0qF7ylZGRAQcHB1y7dg0A0K9fP4PXt7e31/7f09MTnp6eZYpfk+QU7R8CQNvnpnAiVBwrKytt2/nAgQPx5JNPokePHnBzc8PAgQN1yjZu3BiNGzcGoE42pk6diqCgIERFRcHGxgYFBQV444038OKLL+r0C6lMosgw2TNnzsDHxwdNmzYFAOzevRv379/HL7/8or3n/v7+6Nu3r/YYd3d3nb4L165dw//+9z+4uroavObt27d1Xhv6WWrevLm2H4zmgd+iRQu9cq1atcLvv/+O7OxsZGVlITMzE23bti31cwP638OaZCstLa3E4z744AOsWLEC165d0w7vHTlyJPr27Yvp06dj4MCBsLCwgIeHB3bt2oUXX3wRTz/9NAD19+3KlSsxYcIEg0ODhRDl7ldDpsMEg6pE0R92lUoFSZLw22+/QS6X65Uv/Etj2bJlmDhxIn766Sf88ccfeOONNxAWFoZjx46hYcOG5Ypn9OjRmDRpEs6ePYuAgABs374dTz75JFxcXLRlli5digULFmDy5Ml477334OTkBJlMhpCQEJ1OdhVh6LOXtF/zILt+/TqefPJJtGzZEsuXL4ePjw+srKzw66+/4pNPPtHGp/l306ZN8PDw0DufhcWjXwEPHjzQ1nyURnMuJycnKBQKg8M7Nfu8vLyMOmdh3bt3h6enJzZv3qyXYBQ1YsQIrF27Fn///TeCg4OxceNGREVF4YsvvtAmtBr37t1DbGws3NzcYGtrW+a4itIke2lpaTrfi3fu3NH53LGxsWjRooVOQte1a1edc8XHx+skriqVCk899RT++9//Grx28+bNKxy/KZT2vVqc1atXo1+/fnoJwuDBgxEaGorY2Fj4+fkBAJ544gncuHED58+fR3Z2Njp06IB///0XgOH7kJaWpvOzTObBBIMqxbVr19CkSRPt6+joaKhUKu3ogGbNmkEIgSZNmhj1i7Jdu3Zo164d5s+fjyNHjqBHjx5Ys2YNlixZAkA/gSnN0KFDMW3aNG0zydWrV/V68e/YsQN9+/bFunXrdPanp6eb/ZfXzz//jNzcXOzatUvnL8jCTR6A+j4D6jkXihvpoLFt2zZMmjTJqOtrHh4ymQzt2rUzOJvj8ePH0bRpU9SvX9+ocxaVk5NjVMKjqSLXlI2Li0N+fj569OihV3bjxo3YuHEjfvzxR5PMmdKyZUsAQExMDNq1a6fdb29vrxO7h4cH4uLiUFBQoE3qik5EtnbtWp2vUbNmzZCVlVXq101DU1tV2NWrV7U/c5pan6ioKL1yV65cgYuLC+rVqwcbGxvY29sbHIFiSsnJyXoTagFAfn4+AOg0BwLqRCYgIED7et++fQBg8P7ExMSgQ4cOJoyWyoN9MKhSrFq1Suf1ypUrAQD9+/cHoO4HIZfLsWjRIr2/dIQQ2rbtzMxMvV807dq1g0wm06mWr1evnlFDGjUcHR0RHByM7du3Y+vWrbCystJ74Mjlcr3Yvv/+eyQkJBh9ncqi+auxcHwZGRlYv369Trng4GDY29tj6dKl2l/chRWuki9PHwxAXYNw8uRJnSQjKioKf/75J55//nmdsleuXEFcXJz2dXZ2tsG+NT/88APS0tJ0+gAUN/Rx3bp1kCRJO1nV6NGj8eOPP+ptADBgwAD8+OOPJptOulOnTrCystJLsFq1aoWrV69qvyeffvpp3Lt3D6+++iquXbuGM2fOYMqUKZAkCVevXsW0adPw22+/YcGCBdpzjBw5EkePHsXvv/+ud9309HS9n4udO3fqfG+eOHECx48f1/7MeXp6IiAgAN98843Oz8qFCxfwxx9/YMCAAQDUSePQoUPx888/G0wcS6uZMFbz5s2xd+9e7c86oO7rtH37dtSvX1+bHBuSkpKCDz/8EO3bt9dLMDIyMnD9+nV0797dJHFS+bEGgypFTEwMBg8ejGeeeQZHjx7Ft99+i7Fjx2r/qmjWrBmWLFmCuXPnIjY2FkOHDkX9+vURExODH3/8EVOnTsXs2bPx559/YsaMGXj++efRvHlzFBQUYNOmTZDL5Xjuuee01+vUqRP27duH5cuXw8vLC02aNCn1ITJq1Ci88MILWL16NYKDg/XG4w8cOBCLFy/GpEmT0L17d5w/fx6bN2/Wtqub09NPPw0rKysMGjQI06ZNQ1ZWFtauXQs3Nzed5gp7e3t8/vnnePHFF/HYY49h9OjRcHV1RVxcHHbv3o0ePXpo5/0oTx8MAHjttdewdu1aPPvss5g9ezYsLS2xfPlyuLu7480339Qp26pVK/Tu3VvbmfHatWsICgrCqFGj0LJlS8hkMpw6dQrffvstfH19MXPmTO2x77//Pg4fPoxnnnkGjRo1wt27d/HDDz/g5MmTeP3117XV6S1bttTWLBTVpEkTvUTy559/1s5tkp+fj//973/amrHBgwejffv2xX52a2trPP3009i3bx8WL16s3d+sWTP4+flhw4YNCAkJgYeHB1avXo1p06bhq6++giRJmD17NhITEzFt2jR07doVf/31l05t3n/+8x/s2rULAwcOxMSJE9GpUydkZ2fj/Pnz2LFjB2JjY3Vq0vz8/NCzZ0+8+uqryM3NRXh4OJydnXWaWD766CP0798f3bp1w0svvYQHDx5g5cqVcHBwwLvvvqstt3TpUvzxxx/o3bs3pk6dilatWiExMRHff/89Dh06pPezUh5z5szBCy+8gMDAQEydOhU2NjbYsmULTp8+jSVLluh0fu7duze6desGPz8/JCUl4csvv0RWVhZ++eUXbUdsjX379kEIgSFDhlQ4RqogcwxdodpLM0z10qVLYsSIEaJ+/fqiQYMGYsaMGeLBgwd65X/44QfRs2dPUa9ePVGvXj3RsmVLMX36dBEVFSWEEOLGjRti8uTJolmzZsLa2lo4OTmJvn376s0tcOXKFfHEE08IGxsbAcCooYaZmZna8t9++63e+zk5OeLNN98Unp6ewsbGRvTo0UMcPXpU9O7dW/Tu3VtbriLDVIsOvy1udkRDM6Tu2rVLtG/fXlhbW2vnP/j6668FAJ05DTTXCw4OFg4ODsLa2lo0a9ZMTJw4UZw6dcromEsSHx8vRowYIezt7YWdnZ0YOHCguHbtml45ADr3LiUlRUydOlW0bNlS1KtXT1hZWQl/f38REhKiNxvsH3/8IQYOHCi8vLyEpaWlqF+/vujRo4dYv369zjDJ4qCYYaoTJkwodjiuMV/TiIgIIUmS3lDZ9evXCycnJ3Hjxg3tvtTUVPH3339ry548eVLEx8cXe+579+6JuXPnCj8/P2FlZSVcXFxE9+7dxccff6wd3qz5/vvoo4/EsmXLhI+Pj3ZeGM2w8ML27dsnevToIWxsbIS9vb0YNGiQuHTpkl65mzdvivHjxwtXV1ehUChE06ZNxfTp00Vubq728xn6XtV8b2uG35Zkz549onfv3sLFxUVYWVmJdu3aiTVr1uiVmzVrlmjatKlQKBTC1dVVjB07ttiZOkeNGlXizKpUdSQhTFTfRURUBymVSrRu3RojR47UjkoC1E0JgwYNQlRUFH7++edia1V++eUX9OnTx6iF0gyJjY1FkyZN8NFHH5U4Y2ldkJSUhCZNmmDr1q2swagG2AeDiKgC5HI5Fi9ejFWrVunM3SFJErZu3YoWLVogICAAr732Gvbu3YuYmBhER0fj+++/R//+/TFs2DD8+eefZvwEtUd4eDjatWvH5KKaYA0GkQnl5eXh7t27JZZxcHAwam4Iqh1UKhU2btyIjz/+WGf9FwsLCwQHB2PRokXo1KlTuc/PGgyqrtjJk8iEjhw5ojN5kiHr168vdjE2qn1kMhkmTpyIiRMnIiEhAXFxcZDL5WjRokW5J4sjqglYg0FkQmlpaTh9+nSJZdq0aVOu0RpERDUJEwwiIiIyOXbyJCIiIpOrc30wVCoV/v33X9SvX5+L4RAREZWBEAL37t2Dl5eX3iRnRdW5BOPff//VW6mSiIiIjBcfH1/qYpN1LsHQLLwUHx+vs7IhERERlSwzMxM+Pj5GLWJY5xIMTbOIvb09EwwiIqJyMKaLATt5EhERkckxwSAiIiKTY4JBREREJlfn+mAQEZFpCCFQUFAApVJp7lDIhCwtLSGXyyt8HiYYRERUZnl5eUhMTMT9+/fNHQqZmCRJaNiwIezs7Cp0HiYYRERUJiqVCjExMZDL5fDy8oKVlRUnLqwlhBBISUnBrVu34O/vX6GaDCYYJqBUAgcPAomJgKcn0KsXYILaJSKiaikvLw8qlQo+Pj6wtbU1dzhkYq6uroiNjUV+fj4TDHOKiABmzgRu3Xq0r2FDYMUKYPhw88VFRFTZSpsqmmomU9VG8bujAiIigBEjdJMLAEhIUO+PiDBPXERERObGBKOclEp1zYWhxe41+0JC1OWIiIjqGiYY5XTwoH7NRWFCAPHx6nJERGSYUgkcOABs2aL+tyb9Uebr64vw8HBzh1FtsQ9GOSUmmrYcEVFdY44+bH369EFAQIBJEoOTJ0+iXr16FQ+qlmINRjl5epq2HBFRXVJd+7BpJg8zhqurK0fRlIAJRjn16qXOtIvrbCtJgI+PuhwRUV2RnV38lpOjLmNMH7aZM3WbS4o7Z1lMnDgRf/31F1asWAFJkiBJEjZs2ABJkvDbb7+hU6dOUCgUOHToEK5fv44hQ4bA3d0ddnZ26NKlC/bt26dzvqJNJJIk4auvvsKwYcNga2sLf39/7Nq1y6jYDhw4AEmS8Pvvv6Njx46wsbFBv379cPv2bfz2229o1aoV7O3tMXbsWJ3Jzfbs2YOePXvC0dERzs7OGDhwIK5fv65z7vj4eIwcORKOjo5wcnLCkCFDEBsbW7abVw5MMMpJLldX4xmiSTrCwzkfBhHVLXZ2xW/PPacuY0wftlu3dPuw+foaPmdZrFixAt26dcOUKVOQmJiIxMRE+Pj4AADmzJmDDz74AJcvX0b79u2RlZWFAQMGIDIyEv/88w+eeeYZDBo0CHFxcSVeY9GiRRg5ciT+97//YcCAARg3bhzu3r1rdIzvvvsuPvvsMxw5ckSbGISHh+O7777D7t278ccff2DlypXa8tnZ2QgNDcWpU6cQGRkJmUyGYcOGQaVSAQDy8/MRHByM+vXr4+DBgzh8+DDs7OzwzDPPIC8vr2w3sKxEHZORkSEAiIyMDJOc74cfhGjQQAj1j4R68/FR7yciqo0ePHggLl26JB48eKD3XuHfhUW3AQPUZb77ruRymu277x6d18XFcJmy6t27t5g5c6b29f79+wUAsXPnzlKPbdOmjVi5cqX2dePGjcUnn3xS6LNDzJ8/X/s6KytLABC//fZbqefWxLFv3z7tvrCwMAFAXL9+Xbtv2rRpIjg4uNjzpKSkCADi/PnzQgghNm3aJFq0aCFUKpW2TG5urrCxsRG///67wXOU9PUtyzOUnTwraPhwICsLmDABaNsWWLmSM3kSUd2VlVX8e5rfi+Xpw1bZNfqdO3fWeZ2VlYV3330Xu3fvRmJiIgoKCvDgwYNSazDat2+v/X+9evVgb2+P27dvGx1H4ePd3d1ha2uLpk2b6uw7ceKE9vW1a9fwzjvv4Pjx40hNTdXWXMTFxaFt27Y4d+4coqOjUb9+fZ3r5OTk6DWlmBoTDBOwtlb/6+wM9Olj1lCIiMzKmEEVmj5sCQmG+2FIkvr9wn3YKnuwRtHRILNnz8bevXvx8ccfw8/PDzY2NhgxYkSpzQqWlpY6ryVJ0j70jVH4eEmSSj3foEGD0LhxY6xduxZeXl5QqVRo27atNs6srCx06tQJmzdv1ruWq6ur0XGVh1n7YPz9998YNGgQvLy8IEkSdu7cWWL5xMREjB07Fs2bN4dMJkNISEiVxFmajh3V/S1mzjR3JERE1V/hPmxFO8pXdh82Kysro5aXP3z4MCZOnIhhw4ahXbt28PDwqJKOkWVx584dREVFYf78+XjyySfRqlUrpKWl6ZR57LHHcO3aNbi5ucHPz09nc3BwqNT4zJpgZGdno0OHDli1apVR5XNzc+Hq6or58+ejQ4cOlRyd8fz91cnFsGHmjoSIqGYYPhzYsQPw9tbd37Chen9lzYPh6+uL48ePIzY2VqdJoSh/f39ERETg7NmzOHfuHMaOHVummoiq0KBBAzg7O+PLL79EdHQ0/vzzT4SGhuqUGTduHFxcXDBkyBAcPHgQMTExOHDgAN544w3cKqmnrQmYNcHo378/lixZgmFGPpl9fX2xYsUKjB8/vtIzLyIiqlzDh6v7VuzfD3z3nfrfmJjKXShy9uzZkMvlaN26NVxdXYvtU7F8+XI0aNAA3bt3x6BBgxAcHIzHHnus8gIrB5lMhq1bt+L06dNo27YtZs2ahY8++kinjK2tLf7++280atQIw4cPR6tWrfDSSy8hJycH9vb2lRqfJIShFrCqJ0kSfvzxRwwdOtSo8sbOxpabm4vc3Fzt68zMTPj4+CAjI8NkN/fePeDcOXV1XrduJjklEVG1lZOTg5iYGDRp0gTWmk5oVGuU9PXNzMyEg4ODUc/QWj8PRlhYGBwcHLSbZsyzKV2+rO6MNGaMyU9NRERUI9X6BGPu3LnIyMjQbvHx8Sa/hpWV+t/KnrOEiIhqpldeeQV2dnYGt1deecXc4VWKWj9MVaFQQKFQVPI11P8ywSAiIkMWL16M2bNnG3yvsvtCmEutTzCqgqYGo1BXDyIiIi03Nze4ubmZO4wqZdYEIysrC9HR0drXMTExOHv2LJycnNCoUSPMnTsXCQkJ2Lhxo7bM2bNntcempKTg7NmzsLKyQuvWras6fC02kRAREekya4Jx6tQp9O3bV/taM353woQJ2LBhAxITE/WGEHXs2FH7/9OnT+O7775D48aNzToBSuEmEiGKX2GViIiorjBrgtGnTx+UNEp2w4YNevuqyahaHZoaDADIz9d9TUREVBexD4YJ2NoC77+vTixYe0FERMQEwySsrIC33zZ3FERERNVHrZ8Hg4iIqi+lEDiQloYtyck4kJYGZTVsBi/M19e31BmkSY01GCbyv/8BDx4AHTo8Wr6diIiKF5GSgpnR0bhVaIx/Q4UCK/z8MLySlxKnyscaDBN54gng8ceBYtbNISKiQiJSUjDi4kWd5AIAEnJzMeLiRUSkpJgpMjIVJhgmwsm2iKguE0IgW6k0asssKMAb167BUGOIZt/M6GhkFhQYdT5jRxd++eWX8PLy0lt2fciQIZg8eTKuX7+OIUOGwN3dHXZ2dujSpQv27dtX7nsiSRK++OILDBw4ELa2tmjVqhWOHj2K6Oho9OnTB/Xq1UP37t1x/fp17THGxJCbm4vZs2fD29sb9erVQ2BgIA4cOFDuOCsLm0hMhNOFE1Fddl+lgt3BgyY5lwBwKzcXDocOGVU+q1cv1JPLSy33/PPP4/XXX8f+/fvx5JNPAgDu3r2LPXv24Ndff0VWVhYGDBiA999/HwqFAhs3bsSgQYMQFRWFRo0aleuzvPfee1i+fDmWL1+Ot956C2PHjkXTpk0xd+5cNGrUCJMnT8aMGTPw22+/qT+LETHMmDEDly5dwtatW+Hl5YUff/wRzzzzDM6fPw9/f/9yxVkZWINhIpzNk4ioemvQoAH69++P7777Trtvx44dcHFxQd++fdGhQwdMmzYNbdu2hb+/P9577z00a9YMu3btKvc1J02ahJEjR6J58+Z46623EBsbi3HjxiE4OBitWrXCzJkzdWofSoshLi4O69evx/fff49evXqhWbNmmD17Nnr27In169eXO87KwBoME9HUYLCJhIjqIluZDFm9ehlV9u/0dAw4f77Ucr+2a4cnHB2Nuraxxo0bhylTpmD16tVQKBTYvHkzRo8eDZlMhqysLLz77rvYvXs3EhMTUVBQgAcPHujNKF0W7du31/7f3d0dANCuXTudfTk5OcjMzIS9vX2pMZw/fx5KpRLNmzfXuU5ubi6cnZ3LHWdlYIJhIqzBIKK6TJIko5opAOBpJyc0VCiQkJtrsB+GBPVokqednCA38eyFgwYNghACu3fvRpcuXXDw4EF88sknAIDZs2dj7969+Pjjj+Hn5wcbGxuMGDECeRX4xW5paan9v/Twsxjap+kXUloMWVlZkMvlOH36NORF7rednV2546wMTDBMhH0wiIiMI5ckrPDzw4iLFyEBOkmGJp0I9/MzeXIBANbW1hg+fDg2b96M6OhotGjRAo899hgA4PDhw5g4cSKGDRsGQP0wr+p1rkqLoWPHjlAqlbh9+zZ6GVljZC7sg2EikycDCxcCfn7mjoSIqPob7uqKHW3awFvz19lDDRUK7GjTplLnwRg3bhx2796Nr7/+GuPGjdPu9/f3R0REBM6ePYtz585h7NixeiNOKltpMTRv3hzjxo3D+PHjERERgZiYGJw4cQJhYWHYvXt3lcZaGtZgmMi0aeaOgIioZhnu6oohLi44mJ6OxLw8eFpZoZejY6XUXBTWr18/ODk5ISoqCmPHjtXuX758OSZPnozu3bvDxcUFb731FjIzMys1lqKMiWH9+vVYsmQJ3nzzTSQkJMDFxQWPP/44Bg4cWKWxlkYS1XF50kqUmZkJBwcHZGRkwN7e3tzhEBHVODk5OYiJiUGTJk1gzamLa52Svr5leYayBsNE/v0XSEsD3N0BFxdzR0NERGRe7INhIiEhQNu2wNat5o6EiIgq2+bNm2FnZ2dwa9OmjbnDqxZYg2EiHEVCRFR3DB48GIGBgQbfKzwMtS5jgmEiXIuEiKjuqF+/PurXr2/uMKo1NpGYCCfaIqK6po6NEagzTPV1ZYJhImwiIaK6QtMEcP/+fTNHQpVBM2to0ZlCy4pNJCbCJhIiqivkcjkcHR1x+/ZtAICtra12ymuq2VQqFVJSUmBrawsLi4qlCEwwTIQ1GERUl3h4eACANsmg2kMmk6FRo0YVThqZYJhIz57A7NlAjx7mjoSIqPJJkgRPT0+4ubkhPz/f3OGQCVlZWUFWhhVqi8MEw0SCg9UbEVFdIpfLK9xWT7WTWTt5/v333xg0aBC8vLwgSRJ27txZ6jEHDhzAY489BoVCAT8/P2zYsKHS4yQiIqKyMWuCkZ2djQ4dOmDVqlVGlY+JicGzzz6Lvn374uzZswgJCcHLL7+M33//vZIjLV12NhAbq54ynIiIqK4zaxNJ//790b9/f6PLr1mzBk2aNMGyZcsAAK1atcKhQ4fwySefINjM7RNbtwIvvwwMHAj8/LNZQyEiIjK7GjUPxtGjRxEUFKSzLzg4GEePHi32mNzcXGRmZupslYETbRERET1SoxKMpKQkuLu76+xzd3dHZmYmHjx4YPCYsLAwODg4aDcfH59KiU0zTJXzYBAREdWwBKM85s6di4yMDO0WHx9fKddhDQYREdEjNWqYqoeHB5KTk3X2JScnw97eHjY2NgaPUSgUUGiqFyoRJ9oiIiJ6pEbVYHTr1g2RkZE6+/bu3Ytu3bqZKaJHOFU4ERHRI2ZNMLKysnD27FmcPXsWgHoY6tmzZxEXFwdA3bwxfvx4bflXXnkFN27cwH//+19cuXIFq1evxvbt2zFr1ixzhK+DTSRERESPmLWJ5NSpU+jbt6/2dWhoKABgwoQJ2LBhAxITE7XJBgA0adIEu3fvxqxZs7BixQo0bNgQX331ldmHqAKAtzfw6qvAw+n5iYiI6jRJmGrh9xoiMzMTDg4OyMjIgL29vbnDISIiqjHK8gytUX0wiIiIqGZggmEiSiWQksKpwomIiIAaNky1Ovv3X6BRI3VnT44kISKiuo41GCZSeBRJ3erVQkREpI8JhokUnsuroMB8cRAREVUHTDBMRFODAbCJhIiIiAmGiRROMDjZFhER1XVMMEzEwgKQPbybTDCIiKiuY4JhQlyPhIiISI3DVE1o/Hj1fBjFLOxKRERUZzDBMKEvvjB3BERERNUDm0iIiIjI5JhgmFBODpCRAeTnmzsSIiIi82KCYUKPPQY4OgJHjpg7EiIiIvNigmFCHEVCRESkxgTDhAqvR0JERFSXMcEwIc16JEwwiIiormOCYUJsIiEiIlJjgmFCbCIhIiJSY4JhQmwiISIiUuNMnibUq5d6mvAmTcwdCRERkXkxwTChN980dwRERETVA5tIiIiIyOSYYJiYUgkUFJg7CiIiIvOqFgnGqlWr4OvrC2trawQGBuLEiRPFls3Pz8fixYvRrFkzWFtbo0OHDtizZ08VRlu8WbMACwtg4UJzR0JERGReZk8wtm3bhtDQUCxcuBBnzpxBhw4dEBwcjNu3bxssP3/+fHzxxRdYuXIlLl26hFdeeQXDhg3DP//8U8WR67O0VP/LeTCIiKiuM3uCsXz5ckyZMgWTJk1C69atsWbNGtja2uLrr782WH7Tpk14++23MWDAADRt2hSvvvoqBgwYgGXLllVx5Po4DwYREZGaWROMvLw8nD59GkFBQdp9MpkMQUFBOHr0qMFjcnNzYW1trbPPxsYGhw4dKrZ8ZmamzlZZOJMnERGRmlkTjNTUVCiVSri7u+vsd3d3R1JSksFjgoODsXz5cly7dg0qlQp79+5FREQEEhMTDZYPCwuDg4ODdvPx8TH559DgRFtERERqZm8iKasVK1bA398fLVu2hJWVFWbMmIFJkyZBJjP8UebOnYuMjAztFh8fX2mxsQaDiIhIzawJhouLC+RyOZKTk3X2Jycnw8PDw+Axrq6u2LlzJ7Kzs3Hz5k1cuXIFdnZ2aNq0qcHyCoUC9vb2OltlYQ0GERGRmlkTDCsrK3Tq1AmRkZHafSqVCpGRkejWrVuJx1pbW8Pb2xsFBQX44YcfMGTIkMoOt1RNmgADBwKdO5s7EiIiIvMy+1ThoaGhmDBhAjp37oyuXbsiPDwc2dnZmDRpEgBg/Pjx8Pb2RlhYGADg+PHjSEhIQEBAABISEvDuu+9CpVLhv//9rzk/BgCgf3/1RkREVNeZPcEYNWoUUlJS8M477yApKQkBAQHYs2ePtuNnXFycTv+KnJwczJ8/Hzdu3ICdnR0GDBiATZs2wdHR0UyfgIiIiIqShBDC3EFUpczMTDg4OCAjI6NS+2MQERHVNmV5hta4USTV2R9/ANbWQNeu5o6EiIjIvJhgmJBcrh6impNj7kiIiIjMiwmGCXEeDCIiIjUmGCbEtUiIiIjUmGCYECfaIiIiUmOCYUJsIiEiIlJjgmFCbCIhIiJSM/tEW7VJ/fpAnz7qf4mIiOoyJhgm5O4O7N9v7iiIiIjMj00kREREZHJMMIiIiMjkmGCYUEEB4OKi7oNx9665oyEiIjIf9sEwIblcnVgIwZEkRERUt7EGw4Qk6dFkW5wLg4iI6jImGCbGuTCIiIiYYJgcZ/MkIiJigmFyXI+EiIiICYbJsQaDiIiIo0hMrnNnwNsbsLU1dyRERETmwwTDxLZvN3cERERE5scmEiIiIjI5JhhERERkckwwTGzMGMDTE4iIMHckRERE5sMEw8TS04GkJODePXNHQkREZD7VIsFYtWoVfH19YW1tjcDAQJw4caLE8uHh4WjRogVsbGzg4+ODWbNmIScnp4qiLRln8iQiIqoGCca2bdsQGhqKhQsX4syZM+jQoQOCg4Nx+/Ztg+W/++47zJkzBwsXLsTly5exbt06bNu2DW+//XYVR24YJ9oiIiKqBgnG8uXLMWXKFEyaNAmtW7fGmjVrYGtri6+//tpg+SNHjqBHjx4YO3YsfH198fTTT2PMmDGl1npUFU60RUREZOYEIy8vD6dPn0ZQUJB2n0wmQ1BQEI4ePWrwmO7du+P06dPahOLGjRv49ddfMWDAAIPlc3NzkZmZqbNVJjaREBERmXmirdTUVCiVSri7u+vsd3d3x5UrVwweM3bsWKSmpqJnz54QQqCgoACvvPJKsU0kYWFhWLRokcljLw6XayciIqoGTSRldeDAASxduhSrV6/GmTNnEBERgd27d+O9994zWH7u3LnIyMjQbvHx8ZUaX6NGQEAA4OZWqZchIiKq1sxag+Hi4gK5XI7k5GSd/cnJyfDw8DB4zIIFC/Diiy/i5ZdfBgC0a9cO2dnZmDp1KubNmweZTDdnUigUUGiqFarA3LnqjYiIqC4zaw2GlZUVOnXqhMjISO0+lUqFyMhIdOvWzeAx9+/f10si5HI5AEAIUXnBEhERkdHMvthZaGgoJkyYgM6dO6Nr164IDw9HdnY2Jk2aBAAYP348vL29ERYWBgAYNGgQli9fjo4dOyIwMBDR0dFYsGABBg0apE00iIiIyLzMnmCMGjUKKSkpeOedd5CUlISAgADs2bNH2/EzLi5Op8Zi/vz5kCQJ8+fPR0JCAlxdXTFo0CC8//775voIOr76CvjgA2DoUODjj80dDRERkXlIoo61K2RmZsLBwQEZGRmwt7c3+fk/+QQIDQXGjgU2bzb56YmIiMymLM/QGjeKpLrjPBhERERMMEyO82AQERExwTA51mAQERExwTA5rkVCRETEBMPkuJoqEREREwyTc3AA/PwAb29zR0JERGQ+5U4wNm3ahB49esDLyws3b94EAISHh+Onn34yWXA1Ub9+wLVrwPbt5o6EiIjIfMqVYHz++ecIDQ3FgAEDkJ6eDqVSCQBwdHREeHi4KeMjIiKiGqhcCcbKlSuxdu1azJs3T2d67s6dO+P8+fMmC46IiIhqpnIlGDExMejYsaPefoVCgezs7AoHVZNdvaperr1PH3NHQkREZD7lSjCaNGmCs2fP6u3fs2cPWrVqVdGYarSCAuDcOeDiRXNHQkREZD7lWuwsNDQU06dPR05ODoQQOHHiBLZs2YKwsDB89dVXpo6xRuE8GEREROVMMF5++WXY2Nhg/vz5uH//PsaOHQsvLy+sWLECo0ePNnWMNQpn8iQiIjLBaqr3799HVlYW3NzcTBVTpars1VSTkwEPD0CSAKVS/S8REVFtUJZnaLlqMAqztbWFra1tRU9Ta2hqMIRQ98ewtDRvPEREROZQ7gRjx44d2L59O+Li4pBXpD3gzJkzFQ6sptIkGIC6mYQJBhER1UXlGkXy6aefYtKkSXB3d8c///yDrl27wtnZGTdu3ED//v1NHWO1pxQCB9LSsCU5GccepMHDS6BxY3UNBhERUV1UrhqM1atX48svv8SYMWOwYcMG/Pe//0XTpk3xzjvv4O7du6aOsVqLSEnBzOho3Co0bKThDgWW+/nBwcHVjJERERGZT7lqMOLi4tC9e3cAgI2NDe7duwcAePHFF7FlyxbTRVfNRaSkYMTFizrJBQAk5OZixMWLiEhJMVNkRERE5lWuBMPDw0NbU9GoUSMcO3YMgHqGzwoOSqkxlEJgZnQ0DH1azb6Q6Ggo68j9ICIiKqxcCUa/fv2wa9cuAMCkSZMwa9YsPPXUUxg1ahSGDRtm0gCrq4Pp6Xo1F4UJAPG5uTiYnl5lMREREVUX5eqD8eWXX0KlUgEApk+fDhcXFxw+fBiDBw/GK6+8YtIAq6tEI2fSMrYcERFRbVKuBEMmkyEvLw9nzpzB7du3YWNjg6CgIADq9UgGDRpk0iCrI8/C41FNUI6IiKg2KVeCsWfPHrz44ou4c+eO3nuSJEGpVFY4sOqul6MjGioUSMjNNdgPAyrATaZAL0fHKo6MiIjI/MrVB+P111/HyJEjkZiYCJVKpbOVJ7lYtWoVfH19YW1tjcDAQJw4caLYsn369IEkSXrbs88+W56PUm5yScIKPz/DbwoAEvBKgR/knCuciIjqoHIlGMnJyQgNDYW7u3uFA9i2bRtCQ0OxcOFCnDlzBh06dEBwcDBu375tsHxERAQSExO124ULFyCXy/H8889XOJayGu7qih1t2ug1g1imK4CFbdA1l/NgEBFR3VSuBGPEiBE4cOCASQJYvnw5pkyZgkmTJqF169ZYs2YNbG1t8fXXXxss7+TkBA8PD+22d+9e2NramiXBANRJRlTXrtrXP7dtiy7LHwcOunLJdiIiqrPK1Qfjs88+w/PPP4+DBw+iXbt2sCyy4MYbb7xh1Hny8vJw+vRpzJ07V7tPJpMhKCgIR48eNeoc69atw+jRo1GvXj2D7+fm5iK30JM+MzPTqPOWRX0LC9STyZCtUqGlrS2cHCW4uACycqVvRERENV+5EowtW7bgjz/+gLW1NQ4cOACpUD8DSZKMTjBSU1OhVCr1mlrc3d1x5cqVUo8/ceIELly4gHXr1hVbJiwsDIsWLTIqnopwtbJCdk4Obufn4+efK/1yRERE1Vq5/saeN28eFi1ahIyMDMTGxiImJka73bhxw9QxFmvdunVo164duhZqoihq7ty5yMjI0G7x8fGVEovbw1qclPz8Sjk/ERFRTVKuGoy8vDyMGjUKsgq2Abi4uEAulyM5OVlnf3JyMjw8PEo8Njs7G1u3bsXixYtLLKdQKKBQKCoUpzFcNQkGJ9YiIiIqXw3GhAkTsG3btgpf3MrKCp06dUJkZKR2n0qlQmRkJLp161bisd9//z1yc3PxwgsvVDgOU3B7OJLkdn4+PvwQ6NMH2LrVvDERERGZS7lqMJRKJf7v//4Pv//+O9q3b6/XyXP58uVGnys0NBQTJkxA586d0bVrV4SHhyM7OxuTJk0CAIwfPx7e3t4ICwvTOW7dunUYOnQonJ2dy/MRTM61UBNJRhTw11/AM8+YOSgiIiIzKVeCcf78eXTs2BEAcOHCBZ33pDJOLDVq1CikpKTgnXfeQVJSEgICArBnzx5tx8+4uDi9ppioqCgcOnQIf/zxR3nCrxSFm0jsHk6LwdYSIiKqq8qVYOzfv9+kQcyYMQMzZsww+J6h+TZatGhR7ZaFL9xE4vywywcTDCIiqqs4U4OJFG4i0UzsyYm2iIiormKCYSLaGoy8PG2CwRoMIiKqq5hgmIhODYZC3XzDGgwiIqqrmGCYiCbByBcCqKeEtTWnCiciorqLj0ATsZHLYSeXAwDGvpqHBw+A1avNHBQREZGZMMEwIVdOF05ERASACYZJMcEgIiJSY4JhQpqRJAcv5GHgQODtt80cEBERkZmUa6ItMkxTg3HrXj527wbu3zdzQERERGbCGgwT0izZnm2pbiLhMFUiIqqrmGCYkOvDJpJMuXqGrcRE4MABQKk0Y1BERERmwATDhDRNJEcvqWswYmKAvn0BX18gIsKMgREREVUxJhgmdO3kw8m26unOEZ6QAIwYwSSDiIjqDiYYJqJUAl/+38NFSBx0h6lqFn4NCWFzCRER1Q1MMEzk4EHgdpS6BgOO+QB0l5MXAoiPV5cjIiKq7ZhgmEhiIoD0hwmGpQDqFRRfjoiIqJZjgmEinp4A8uXAffV6JOpajGLKERER1XJMMEykVy+gYUM8qsUokmBIEuDjoy5HRERU2zHBMBG5HFixAoUSjEcjSSRJ/W94uLocERFRbccEw4SGDwc6+z0cSVKoBqNhQ2DHDvX7REREdQETDBNr30hdg/HMGHWC4eOjnnCLyQUREdUlTDBMTDObp1frR9OFq1TmjIiIiKjqMcEwMc2S7Q+s8lGvHlBQANy4YeagiIiIqhgTDBPT1GCkFOSjRQv1vitXzBgQERGRGVSLBGPVqlXw9fWFtbU1AgMDceLEiRLLp6enY/r06fD09IRCoUDz5s3x66+/VlG0JdPUYKTk5aFlS/U+JhhERFTXWJg7gG3btiE0NBRr1qxBYGAgwsPDERwcjKioKLi5uemVz8vLw1NPPQU3Nzfs2LED3t7euHnzJhwdHas+eAO0NRj5+WjZEnByUjeTEBER1SWSEEKUXqzyBAYGokuXLvjss88AACqVCj4+Pnj99dcxZ84cvfJr1qzBRx99hCtXrsDy4cO8JLm5ucjNzdW+zszMhI+PDzIyMmBvb2+6D/JQfE4OGh07BktJQnb3J2BpKZn8GkREROaQmZkJBwcHo56hZm0iycvLw+nTpxEUFKTdJ5PJEBQUhKNHjxo8ZteuXejWrRumT58Od3d3tG3bFkuXLoWymGVKw8LC4ODgoN18fHwq5bNoaGow8oVANlh1QUREdZNZE4zU1FQolUq4u7vr7Hd3d0dSUpLBY27cuIEdO3ZAqVTi119/xYIFC7Bs2TIsWbLEYPm5c+ciIyNDu8XHx5v8cxRmLZej/sPpOlPyDa9HQkREVNtVi06eZaFSqeDm5oYvv/wSnTp1wqhRozBv3jysWbPGYHmFQgF7e3udrbJpajFu5+dj2jTAzw/4++9KvywREVG1YdYEw8XFBXK5HMnJyTr7k5OT4eHhYfAYT09PNG/eHPJCi3q0atUKSUlJyMvLM3hMVSs8kuTWLeD6deDyZTMHRUREVIXMmmBYWVmhU6dOiIyM1O5TqVSIjIxEt27dDB7To0cPREdHQ1VoesyrV6/C09MTVg8f7OZWuAZDM1SVCQYREdUlZm8iCQ0Nxdq1a/HNN9/g8uXLePXVV5GdnY1JkyYBAMaPH4+5c+dqy7/66qu4e/cuZs6ciatXr2L37t1YunQppk+fbq6PoMfFQj36d19aGmSPpQEywbkwiIioTjH7PBijRo1CSkoK3nnnHSQlJSEgIAB79uzRdvyMi4uDTPYoD/Lx8cHvv/+OWbNmoX379vD29sbMmTPx1ltvmesj6IhIScGO1FQAwPcpKYB3CrBFgTPb/AC4mjc4IiKiKmL2eTCqWlnG8JZVREoKRly8CL0bqgIgAd/5t8GYhkwyiIioZqox82DUJkohMDM6Wj+5ANR3WQCzrkdDWbfyOSIiqqOYYJjIwfR03Co0Y6geGZAscnEwPb3KYiIiIjIXJhgmkmjkENnIc3koZtJRIiKiWoMJhol4GjlEdkmIFXx9gYiIyo2HiIjInJhgmEgvR0c0VChQ7NJmKgDJCuC8IxISgBEjmGQQEVHtxQTDROSShBV+fgCgn2Q8HEWCVX6ASoKmn2dICNhcQkREtRITDBMa7uqKHW3awFuh0H0jVQEsbAMcfDREVQggPh44eLCKgyQiIqoCTDBMbLirK2Iffxxv3+kAPHh4exe21kkuCktMrMLgiIiIqggTjEoglyQ85dwAON1AvSMgo9iynp5VFBQREVEVYoJRSXr1AhxiHiYYHdP03pckwMdHXY6IiKi2YYJRSeRyYOFgR/WLdhmAhUqvTHi4uhwREVFtwwSjEs0cXA/2KkvARgW0vKfdb2MD7NgBDB9uxuCIiIgqEROMSiSTJDzt7ggAmLwyDUuWAP36AZs3M7kgIqLajQlGJevn6AgAiHFMx7x5QGQkMGyYeWMiIiKqbEwwKlnfhwnGofR0fJOYiANpaVxRlYiIaj0LcwdQ213MzoYMQD6AiVFRAAB3mQJPXvbDxpdd2cmTiIhqJdZgVKKIlBQ8f+kSio4fSS7IxXfNL2L06hQcOMDpwomIqPZhglFJlEJgZnQ0DDaGyAAIYIdbNPo+Kbi6KhER1TpMMCrJwfR03MrNLb6ADIB7LtAunaurEhFRrcMEo5Ik5uUZV9A5j6urEhFRrcMEo5J4WlkZV7BBHiATXF2ViIhqFSYYlaSXoyMaKhSQSis44zqw5RjQKwUAV1clIqLagQlGJZFLElb4+QFA6UmGSy6w6CLQKwWXLoEjS4iIqMarFgnGqlWr4OvrC2trawQGBuLEiRPFlt2wYQMkSdLZrK2tqzBa4w13dcWONm3grVCUXPDhqBJMj8aSpQJ9+4IjS4iIqEYze4Kxbds2hIaGYuHChThz5gw6dOiA4OBg3L59u9hj7O3tkZiYqN1u3rxZhRGXzXBXV8Q+/jg+adas5IKFRpUA4MgSIiKq0cyeYCxfvhxTpkzBpEmT0Lp1a6xZswa2trb4+uuviz1GkiR4eHhoN3d39yqMuOzkkgR3Yzt9OqtHn3BkCRER1WRmTTDy8vJw+vRpBAUFaffJZDIEBQXh6NGjxR6XlZWFxo0bw8fHB0OGDMHFixeLLZubm4vMzEydzRyMHlVy51E5jiwhIqKayqwJRmpqKpRKpV4NhLu7O5KSkgwe06JFC3z99df46aef8O2330KlUqF79+64deuWwfJhYWFwcHDQbj4+Pib/HMYodVSJCkCyAjjvqPcWR5YQEVFNY/YmkrLq1q0bxo8fj4CAAPTu3RsRERFwdXXFF198YbD83LlzkZGRod3i4+OrOGK1UkeVSABW+QEq/Xc9PSs1NCIiIpMza4Lh4uICuVyO5ORknf3Jycnw8PAw6hyWlpbo2LEjoqOjDb6vUChgb2+vs5lLiaNKkq2AexZAv2SgQxogU3fCcHJS98FgPwwiIqpJzJpgWFlZoVOnToiMjNTuU6lUiIyMRLdu3Yw6h1KpxPnz5+FZQ/7M14wq2d+hA75r1Qpv3WsLPJAAjzzgk3PAgstA+Dnt5Ft37wJBQRy2SkRENYvZm0hCQ0Oxdu1afPPNN7h8+TJeffVVZGdnY9KkSQCA8ePHY+7cudryixcvxh9//IEbN27gzJkzeOGFF3Dz5k28/PLL5voIZSaXJPRp0ABj3N3R9XEBWBtYc7XQ5FsAh60SEVHNYmHuAEaNGoWUlBS88847SEpKQkBAAPbs2aPt+BkXFweZ7FEelJaWhilTpiApKQkNGjRAp06dcOTIEbRu3dpcH6HcNEu6G+yUIYO64+f0aOCwC4RKgiSph60OGQLI5VUbKxERUVlIQggDfz7XXpmZmXBwcEBGRoZZ+2MAwIG0NPQ9d670giEdgHMNtC8/+QR4/XUmGUREVLXK8gw1exNJXVaWJd0LmzWLfTKIiKh6Y4JhRmVd0r0w9skgIqLqjAmGGZV3SXdAPcunEMArrwCbN3MFViIiql6YYJhReZd0LywlBXjhBXAFViIiqlaYYJhZmZd0fzMK6PhwIi6ZUE/K9XByrluJgs0mRERULXAUSTWhFAIrb93CrOvXjTsg4+EIY4eCR/tuK4BVfvCJcUVMDEeZEBGRaXEUSQ1UpiXdAcC+QL0V5pILvHsR8b4pWLmSfTKIiMh8mGBUI0aPKgHUnTaKdtzQNKNMj8asNwX7ZBARkdkwwahGjB5VUhIZAPdcoF06h7ISEZHZMMGoRso0qqQ0znnaoaxTpgCRkWwyISKiqsMEo5oxelRJae48am7hiqxERFTVzL7YGekb7uqKIS4uOJCWhpGXLuFuQUHpB2moAKQogPOO6mGs7dLVU43fscKtC4547jkJixYB/v6ApyfQqxdHmxARkelxmGo1F5GSghEXLwJQ998skabAwjbqf2dEA265j96/rQA+8wMOump3NWwIrFgBDB9uspCJiKiW4jDVWqS4JhNnCws4WxSpgBIAvvNR11osugi45uq+b2A20Fu3gOeeUy+gxunGiYjIVFiDUUMohcDB9HQk5uXB08oKvRwdAQAH09Px0+E8hOdGA875pZ9IBSDdEljtB6RaqZtSVI+6lLJGg4iIilOWZygTjFogIiUFz124WL6hJ0WaTaSH59ixg0kGERHpYhNJHaIUAjOjo8s/rrVIswmHthIRkSkwwajhDqan41ZubukFi1No9k/IHlVmlWVoq1IIHEhLw5bkZBxIS4OyblWKERGRARymWsMl5uVV/CSFZv/EeccyDW2NSEnBzOhonSSnoUKBFX5+GO7qauBiRERUFzDBqOHKtH5JaXqkAm9fMTi0deHCR8mCtzcwdSqQ1i4F4Q0u6p0mITcXIy5exI42bZhkEBHVUezkWcMphYDvsWNIyM0tfZ6M0mhOULg/h+rh64VtdObPgEwAW46ph8Ia6P8hQV2TEfP445BLFZ74nIiIqgF28qxDjFm/5A1vb7haWpbeD7SUFVoL99FAu3R1TUcxJxUA4nNz8W5EOufXICKqg5hg1ALFTcblo1DghzZtsMLfH2uaNwdQzsEmmj4aE2KADmnqRMPZuL4fS1bnoW9froNCRFTXsImkFjE0GVfh5glDHTLL5bYC+MUTmBxbetmQDsC5BiaZX6O0z0dERJWrxjWRrFq1Cr6+vrC2tkZgYCBOnDhh1HFbt26FJEkYOnRo5QZYQ8glCX0aNMAYd3f0adBA7+E73NUVsY8/jk+aNavYhVxygUmxQIZF8QukqADctQRc8oAOaRCSKHF+DaVSPVX5li2PpiwvvG/xgRT4Hj2GvufOYezly+h77hx8jx1DREqK/rWJiMjszD6KZNu2bQgNDcWaNWsQGBiI8PBwBAcHIyoqCm5ubsUeFxsbi9mzZ6NXr15VGG3NJ5ckuFd05IkM6gTCUlV8GQmAUz4w/7L69cPRKHcPuiIo6NFIFH9/4No14MuvBBKc0rXDY53+dYSkknDnDtSTgC26CORCp40nITcXz128iJC0NhjSwBW9egGQsZaDiKg6MHsTSWBgILp06YLPPvsMAKBSqeDj44PXX38dc+bMMXiMUqnEE088gcmTJ+PgwYNIT0/Hzp07jbpebW4iMdaBtDT0PXfOdCcUKL1zR3GjUQB1AlHcyq+HXUocraJdnn7s43AenArMiMYdOefkICKqDDWmiSQvLw+nT59GUFCQdp9MJkNQUBCOHj1a7HGLFy+Gm5sbXnrppVKvkZubi8zMTJ2truvl6IiGCkW5Zxc3SAD43htIszTcbKIZjfJmFNAx7dGIFE3tRHErv467WeJoFW0H1HGxuPPGRdyR6Z5HMycHm1KIiKqWWROM1NRUKJVKuLu76+x3d3dHUlKSwWMOHTqEdevWYe3atUZdIywsDA4ODtrNx8enwnHXdMYMbX3OxcX4E0pQJw9P3gYa5JecDDgUAMvPqWslnritrrkwFIgmIXnulnExPJdg8DwC6rVVpv4vGpEHBIfLEhFVkWrRydNY9+7dw4svvoi1a9fCxcgH4Ny5c5GRkaHd4uPjKznKmqG0oa3b2rQpWy2HDOo+F8ZyyQXevVR67YRDgXHncygo/jwScEeei6CQdDRuDCxeXHxnUs7ZQURkGmbt5Oni4gK5XI7k5GSd/cnJyfDw8NArf/36dcTGxmLQoEHafSqVuqOhhYUFoqKi0KzICAmFQgFFkYcoqQ13dcUQF5diO0Wu8PPDiIsXtRUUJmWq1FYFIEcG2JbQ4VTDOQ8J5wUWRqQb7kz6UMOGwIoVlb9cPYfdElFtVi06eXbt2hUrV64EoE4YGjVqhBkzZuh18szJyUF0dLTOvvnz5+PevXtYsWIFmjdvDqtSRkiwk2fZlHXuDAdYIhP5pk9IAOM6k5bkV3egcxrgVmiSME1n0qIdTwGEhABDhugu7qahVAIHDwKJieoF4Lp3B44cefTa4DGFEopr9+9jbWIibhVarM7bygpTvbzgb2NjdMLBJIWIqlJZnqFmTzC2bduGCRMm4IsvvkDXrl0RHh6O7du348qVK3B3d8f48ePh7e2NsLAwg8dPnDiRo0gqmWY59pGXLuFugeEmC83aI8ubNcPIS5cAVEKtR1HlSTiKHmNodItM6Kwo63XHEdOmSNrVZFNTgVmzgFuFuofI5bpNK4WH4Xp6AqmtUjDrejRu5Rk/yVlDKwVW+Bc/AoYr2RJRVSvLM9Ts82CMGjUKKSkpeOedd5CUlISAgADs2bNH2/EzLi4OMlmN6ipS68glCU86OWFtixYYcVG9emrh5EHzvA5/+GDbIUmmmTG0JMUlF6UlHYY6k6oAhFwFrFSA131gYKJOLce/txVY+JkfsLD4h3bRfhsJCcDChQ9faEbKGLp+CW7l5OK5CxexSGoD/0RXnZqRiJQUjLh4US+J40q2rNUhqi7MXoNR1ViDUTGG/mr2USi0yYWGMbUe1VZxtRw7GgKHnYHzjur9hWo5cNEBaJOh/9olF5h+HXAsYXRNSQrN8wGVBG9v4OWpAit7HMNdWTEdZAXgrFJgm/xx9OklVcrkY9X1Ic5aHaLKVaOaSKoaE4yKK8vDRfOXNlAFTSZVJeNhxV/hES5KAIX7XBR9XVEbGwFnGqgTlyEJwIzrpR8T0gHOTQr0Jh/ztlJgap6ftlake0+BI/d0v54AdL7G3es74sghCYmJwDXPFKy10m3uqQ4P8eJqdTTfmXW5VofIVJhglIAJRtUztqOoBHVHRwBIyMurvgmJJjCpyL6SXptKWRKX772BEQbmBync7wSAbGY0VM6PvjbOFuoE6k6hmif5HQWUK9Rzpxhq7invQ7xostrdwQFHMjLKXDOiFAK+x44V+z2m6SMU8/jjkEuSXiddQ51yiUgfE4wSMMEwD82D5KfUVIQnJOgNfS38gAJQbF8PAfUD8G5BgcEERAJgD0tkoAxzctQkZUlc7skBO2XxU6w/KDS8t2hyVHSfJil5IAdsDJ9TAuACBT5JfBzenpLBmpHSVveVQ51DaRiqGTFUg3YwPd2o6e+noxl6pTbE7FmSTiddY4YmV9dmIVOp7Z+vrqmsrycTjBIwwTA/Y/pxlFQGKD4BAYBtrVsj9Pp1JOTmVt9akNpsYyOgQIJsSCJUzoaH4V67fx/v3rxZ6tenaM1IcX0sRri6IvyWkbO+3lYAq5oBGZaP+sxcUveZCXkvD0N6GJcMmaJZqLo81E31+arL56nOquIeVWZfJCYYJWCCUT0Y80NWUpnSkpQa0fejqppVzKW0z1OGz2unssBzMm9shH5SUuaJ4DS1MYWvXaTpqfAvY1P27ShtLhRz9GUx1eerjR1sTZEMmOJrXp5+b5XVF4kJRgmYYNQepf3QGfyFZ2WFKV5eaGZtjVnXryM1v5ImBTNGshWw2wtIsAG89YfHkgHFJSWaiVyLJg7GnqeY8wZneeOY3e0Sm9yclY9G7JTUj8OYvkiGHgLl6S9i7AOprH1XSvps5X2olechXlm1AGVNBsrzO6io0u5RWRI3U309S8IEowRMMOqW0mpBqrSWQwUg3RJY7QekWqmHu6oK/ZBrJvh6LA0YH1cVEdVO5qgJ+qwZPE94I/jNDNg1ykOzBlZ4racj5HLo9D0ylgMsMS7ND/fjrPDHcgf82+DREGjPuw561wGA1YfScT0tD9kN7uMPq0QkGPFX8oG0NKP6ruzv0EHb16Xoz1JFHmrlqfUw9piyJiHGJgMdoqPx25YtuDhvHiba2BQbR3FJV3HnNXSPypq4leXr2adBAyMi08cEowRMMKiwkmo50vLzDXZILQ/NL4R3oZ4069o1YO1a6HU0nDIFaOYvMMvzGFJECQvBFaZCDVu2sJKZq6mpSDOL7J4FbOsBWbIKzgNTyhBoKVM96kfYF7pOMffgmayGCLZzxrQeDjielYEfUlLw2b//lhrCCBcXHMvM1PtrfnmzZkjIzcWs66UPm56OZljesyGsLHQT/OJ+tkIaNsQQZ2eDfxSUdkxqfj5mXb9udOJSlmTgva+/xvxNm/Deiy/incmTdd7T/K5Y2LgxViYklHkOIM2DvyyzJxdOSrYkJ2Ps5culXue7Vq0wpsgq5sZiglECJhhUVFn7ehgcxgnd0Q9FXxucjKyEqu9ia1eKmwTMlA9UQ6NIahpzJF2Gml2Ait/H0ppzynMdU8/TYiT5HQXesGiGhg4WWKi6ZFTypUlknCwsTDJxX9HEpbQamKL+efllBFy/jn/8/PDY2rUViqWo71q1gkImK9NMyIVrI1iDYWZMMKisDCUgAEqcv6G88zkUZuwwzuXNmpU8akYFIN1CvTBA/RKWtddIt1CXcSjlF7kKwH0ZYGfESrbGMnXtQ23rOGsqprovZT1PZSTE5eRtaYWp3l7IV6mwJM64Jkm3u3eR/Nxzj15HRCClnA9qQxY1bmzU6KrCCtdGVLc+GGZfi4SoupNLksFsv+i+0l6X1XBXVwxxcTFqIiq5JGHExYuG5xeRAe86tUBaGhAu6ZcpOhW607+OAIC7XulAj1T1ZF0CujUCmmM+aqmeCt0lt3w1BoZqZEz9l3XR2gwmHeVbx8fY85REhmozrCshLw8LY2PLdEzwyZN6r799+mmTxONiYYEVCQllvj2XsrNxIC1N+7thhIuLwf4+mi9VuJ9flQ0dZoJBVI0ZSm4MJS7DXV2xo00bg53fCjfN9EoxUMZGgSm5fvDv6QrP59VNNQBw8GADJCY2wDXJEWsVRVaCTS20zL2Q1LN7lvYgL65552tf9UgazRoubdOBdy8B9sXUtpTlQbi+MTAwCXAr8hddbR8iXB5V9fmry30uRxwDjh1DvkwGS5UK+XI5Bhw7ZpoEQwCp5Wz6WRIXhyVxcXq1m0UV/V1QFdhEQlSLVHR+EWPOe+24Fb6c4YiE+EfHOA9LAabrrnmC21bALyUMw01WAKseJilFaVagLa7mZENjYHhC8UlI4UXiAKBdOp55IQ/Bna3g1TIfb8bodgA0qk+CCkCWHLAv6dd4GTGxqVa8UlLgnpZm8D1JCPwZGgqH+/e1+zLq1UO/Zcsgivn5SW7QAP8WfaBXVnJbTN+cZ7LVHXtf6+mo7WBbEeyDUQImGEQVZ6iDauFVWw0lITJLAVXrdO1wy4Zpjlj+sQRXV/V59EbW9EoBZkTr1D7IUhWwXe+HrF9dS09CFrYBDrrCxwcID9edBlw7df3hPIQveFhz0jqj9Cahd1tXrEmoKM1cKP9aq8/rkM8RQWa0LzQUT/7zT7HvqyQJskKPzKKv9c732GN4atky3Z1Fv+blWWnZ2KSkUKLd0EsqdTp8YzDBKAETDKKqUTQJ6d4dOHKk5Amjih4T2E3gi6Pq+R20c0tIkrbMNc8UfGkVjYTCK7taKTCl0GqxpU1MFREBzJxZcmKjU9tSXGJjzOiOIv1ddOZCKct5S7tOef4qVgFIsXq4qExe6YlOSZOead4zZ7JUjnsw4sABfLFsGRpkZVWoUkEASLOzw7Q338SOPn3U9+SeBbCoDXDOUf0175AGhJc+4kOrIp2qQzpA+p+6aXXHjoolGUwwSsAEg6h2Mcl0zqUkNp6pjroLpPVKgXxmNJTOJTSzGBqNU1Kz0MPz6iU35ZkHQ/NXcv18wzUyxfWHebjCboUTqK0+wOj40s9R+DjN4numOKYCo1Vc09Lw+aw1eO7mH1BBgqwM3S41NRo/9OqFV2fNUo8wKVKjptUvGVhQ+pwV6hPjUWL6vJFr7hT2XivgT3dIknq+nZiY8q8ezFEkRFRnFDfKp0znkAN9+hTeIyGkj+45RwwrnIS4ontPF53VYgPtHPDF4YxHSUmBI94MBRKc0rXNQt53HTH1ZQn+rxpoEgLgE+uKjyQXJOJRcjPtCfWkWMVd57UBjgAezeSZFWeFvcsKNU+dd9RPWoqO1kkpkvgsbKN/zO1CU9vfsVI350y/rlum8Hku2+ufAyg+ufmwlfq1KY5JUQCrHy5oV1yzlyEqICXPAyPi9+B5fI81mAZ73INFid0n1QogR6bcDq/MmI3vh/TUjcVQUnnHqtRz6p0j06J8CcbDawkBxMerv491v98rB2swiIgqSWnriJRnnZHyXLdwjUyWgenH9aatBx5NXf+wjOySI1T5hTr2OgNCJtTDmYs7z8NzaDrYxmfmY0X+dd2an9uFRiQZuK7BRKZoTVDRY4rGYahmqKRanIfndcVtbKg/Dv3v7St13b7f6gdh4r3NSJG5lhxL4Xuz5Vjx/XkMNauUdoyhc2g6OxeK4bvvgDFjjDjeADaRlIAJBhHVdYUTkGvXgHffVe839DQICQGGDDHchwbQPY9ejYyBDrZ5BUJb29KsgRWmdXPE8aMSfvpJXVaSDMRRWgJhjMLnKMOopsWYjzmyD2CpKr4Wo0Amx1LVXCzEe2WLyciOykYdY0TCpLF/f/lrMJhglIAJBhGRLr2OrjCcHJSmojUyhuIoD2dn9b937jza17Ah8NJLwMqVwN27MDpp+QcBaI9zJVYYqACcQwAeQ/EjUIpVWqdiY48p2lfHwDmqug8GEwwiIqq05pqKxpGaCsyapZt0yOXqchqahQL9/Q3XrhT+PBERwIgR6vcLP/00NSeLFqnPc+0asHNNEs4keurEp+nIaWiIqjuScBvGLSLm5ARs3w6kpQEhoUKnr07R5iiDiiZIFx2ANsU3e2n6PXMUSSVigkFEVLOUZ8hzSYytsVGt/wayyRO1r4VcjlwbO+wf8BL6/roOigdZkAplOidmfIPndo4vsQbG0IO+pM9nqOmpJCEhQIMGxjVXlQcTjBIwwSAiIqNqbEaNUmcCQqi3YcOANWsANzfg9m3glVeAH39UZw2SBDz/PJSbt5a5X0pZYzVUq1P0vJVVI8UEowRMMIiIqFQFBerOHJmZgKMj8MUXwMiR+uW2bwemTQPS0wF7e3UHj0JP8qoaKVRVTVpleYZWi0lpV61aBV9fX1hbWyMwMBAnTpwotmxERAQ6d+4MR0dH1KtXDwEBAdi0aVMVRktERLXegwdA06bqWouoKMPJBaDeHxWlLtesGVBorRLg0RwrY8ao/zVVElBZ5zUls0+0tW3bNoSGhmLNmjUIDAxEeHg4goODERUVBTc3N73yTk5OmDdvHlq2bAkrKyv88ssvmDRpEtzc3BAcHGyGT0BERLVO/frAqVPGPbnd3NQdO5TK6vmkNxOzN5EEBgaiS5cu+OyzzwAAKpUKPj4+eP311zFnzhyjzvHYY4/h2WefxXvvlT4GmU0kRERE5VNjmkjy8vJw+vRpBAUFaffJZDIEBQXh6NGjpR4vhEBkZCSioqLwxBNPGCyTm5uLzMxMnY2IiIgql1kTjNTUVCiVSri7644bdnd3R1JSUrHHZWRkwM7ODlZWVnj22WexcuVKPPXUUwbLhoWFwcHBQbv5+PiY9DMQERGRvmrRybOs6tevj7Nnz+LkyZN4//33ERoaigMHDhgsO3fuXGRkZGi3+Pj4qg2WiIioDjJrJ08XFxfI5XIkJyfr7E9OToaHh0exx8lkMvj5+QEAAgICcPnyZYSFhaGPgcnVFQoFFAqFSeMmIiKikpk1wbCyskKnTp0QGRmJoUOHAlB38oyMjMSMGTOMPo9KpUJubm7pBaHutwGAfTGIiIjKSPPsNGZ8iNmHqYaGhmLChAno3LkzunbtivDwcGRnZ2PSpEkAgPHjx8Pb2xthYWEA1H0qOnfujGbNmiE3Nxe//vorNm3ahM8//9yo6927dw8A2BeDiIionO7duwcHB4cSy5g9wRg1ahRSUlLwzjvvICkpCQEBAdizZ4+242dcXBxkskddRbKzs/Haa6/h1q1bsLGxQcuWLfHtt99i1KhRRl3Py8sL8fHxqF+/PiSpjMv9PpSZmQkfHx/Ex8dzqKsJ8b5WHt7bysH7Wjl4XytPRe+tEAL37t2Dl5dXqWXNPg9GTcS5NCoH72vl4b2tHLyvlYP3tfJU5b2tkaNIiIiIqHpjgkFEREQmxwSjHBQKBRYuXMjhrybG+1p5eG8rB+9r5eB9rTxVeW/ZB4OIiIhMjjUYREREZHJMMIiIiMjkmGAQERGRyTHBICIiIpNjglEOq1atgq+vL6ytrREYGIgTJ06YO6QaJSwsDF26dEH9+vXh5uaGoUOHIioqSqdMTk4Opk+fDmdnZ9jZ2eG5557TWxSPSvbBBx9AkiSEhIRo9/G+lk9CQgJeeOEFODs7w8bGBu3atcOpU6e07wsh8M4778DT0xM2NjYICgrCtWvXzBhx9adUKrFgwQI0adIENjY2aNasGd577z2dNS54X43z999/Y9CgQfDy8oIkSdi5c6fO+8bcx7t372LcuHGwt7eHo6MjXnrpJWRlZVUsMEFlsnXrVmFlZSW+/vprcfHiRTFlyhTh6OgokpOTzR1ajREcHCzWr18vLly4IM6ePSsGDBggGjVqJLKysrRlXnnlFeHj4yMiIyPFqVOnxOOPPy66d+9uxqhrlhMnTghfX1/Rvn17MXPmTO1+3teyu3v3rmjcuLGYOHGiOH78uLhx44b4/fffRXR0tLbMBx98IBwcHMTOnTvFuXPnxODBg0WTJk3EgwcPzBh59fb+++8LZ2dn8csvv4iYmBjx/fffCzs7O7FixQptGd5X4/z6669i3rx5IiIiQgAQP/74o877xtzHZ555RnTo0EEcO3ZMHDx4UPj5+YkxY8ZUKC4mGGXUtWtXMX36dO1rpVIpvLy8RFhYmBmjqtlu374tAIi//vpLCCFEenq6sLS0FN9//722zOXLlwUAcfToUXOFWWPcu3dP+Pv7i71794revXtrEwze1/J56623RM+ePYt9X6VSCQ8PD/HRRx9p96WnpwuFQiG2bNlSFSHWSM8++6yYPHmyzr7hw4eLcePGCSF4X8uraIJhzH28dOmSACBOnjypLfPbb78JSZJEQkJCuWNhE0kZ5OXl4fTp0wgKCtLuk8lkCAoKwtGjR80YWc2WkZEBAHBycgIAnD59Gvn5+Tr3uWXLlmjUqBHvsxGmT5+OZ599Vuf+Abyv5bVr1y507twZzz//PNzc3NCxY0esXbtW+35MTAySkpJ07quDgwMCAwN5X0vQvXt3REZG4urVqwCAc+fO4dChQ+jfvz8A3ldTMeY+Hj16FI6OjujcubO2TFBQEGQyGY4fP17ua5t9NdWaJDU1FUqlUrvSq4a7uzuuXLlipqhqNpVKhZCQEPTo0QNt27YFACQlJcHKygqOjo46Zd3d3ZGUlGSGKGuOrVu34syZMzh58qTee7yv5XPjxg18/vnnCA0Nxdtvv42TJ0/ijTfegJWVFSZMmKC9d4Z+L/C+Fm/OnDnIzMxEy5YtIZfLoVQq8f7772PcuHEAwPtqIsbcx6SkJLi5uem8b2FhAScnpwrdayYYZFbTp0/HhQsXcOjQIXOHUuPFx8dj5syZ2Lt3L6ytrc0dTq2hUqnQuXNnLF26FADQsWNHXLhwAWvWrMGECRPMHF3NtX37dmzevBnfffcd2rRpg7NnzyIkJAReXl68r7UEm0jKwMXFBXK5XK/XfXJyMjw8PMwUVc01Y8YM/PLLL9i/fz8aNmyo3e/h4YG8vDykp6frlOd9Ltnp06dx+/ZtPPbYY7CwsICFhQX++usvfPrpp7CwsIC7uzvvazl4enqidevWOvtatWqFuLg4ANDeO/5eKJv//Oc/mDNnDkaPHo127drhxRdfxKxZsxAWFgaA99VUjLmPHh4euH37ts77BQUFuHv3boXuNROMMrCyskKnTp0QGRmp3adSqRAZGYlu3bqZMbKaRQiBGTNm4Mcff8Sff/6JJk2a6LzfqVMnWFpa6tznqKgoxMXF8T6X4Mknn8T58+dx9uxZ7da5c2eMGzdO+3/e17Lr0aOH3jDqq1evonHjxgCAJk2awMPDQ+e+ZmZm4vjx47yvJbh//z5kMt1HkFwuh0qlAsD7airG3Mdu3bohPT0dp0+f1pb5888/oVKpEBgYWP6Ll7t7aB21detWoVAoxIYNG8SlS5fE1KlThaOjo0hKSjJ3aDXGq6++KhwcHMSBAwdEYmKidrt//762zCuvvCIaNWok/vzzT3Hq1CnRrVs30a1bNzNGXTMVHkUiBO9reZw4cUJYWFiI999/X1y7dk1s3rxZ2Nraim+//VZb5oMPPhCOjo7ip59+Ev/73//EkCFDOJyyFBMmTBDe3t7aYaoRERHCxcVF/Pe//9WW4X01zr1798Q///wj/vnnHwFALF++XPzzzz/i5s2bQgjj7uMzzzwjOnbsKI4fPy4OHTok/P39OUzVHFauXCkaNWokrKysRNeuXcWxY8fMHVKNAsDgtn79em2ZBw8eiNdee000aNBA2NraimHDhonExETzBV1DFU0weF/L5+effxZt27YVCoVCtGzZUnz55Zc676tUKrFgwQLh7u4uFAqFePLJJ0VUVJSZoq0ZMjMzxcyZM0WjRo2EtbW1aNq0qZg3b57Izc3VluF9Nc7+/fsN/k6dMGGCEMK4+3jnzh0xZswYYWdnJ+zt7cWkSZPEvXv3KhQXl2snIiIik2MfDCIiIjI5JhhERERkckwwiIiIyOSYYBAREZHJMcEgIiIik2OCQURERCbHBIOIiIhMjgkGERERmRwTDCKqFQ4cOABJkvQWcyMi82CCQURERCbHBIOIiIhMjgkGEZmESqVCWFgYmjRpAhsbG3To0AE7duwA8Kj5Yvfu3Wjfvj2sra3x+OOP48KFCzrn+OGHH9CmTRsoFAr4+vpi2bJlOu/n5ubirbfego+PDxQKBfz8/LBu3TqdMqdPn0bnzp1ha2uL7t276y21TkRVgwkGEZlEWFgYNm7ciDVr1uDixYuYNWsWXnjhBfz111/aMv/5z3+wbNkynDx5Eq6urhg0aBDy8/MBqBODkSNHYvTo0Th//jzeffddLFiwABs2bNAeP378eGzZsgWffvopLl++jC+++AJ2dnY6ccybNw/Lli3DqVOnYGFhgcmTJ1fJ5yeiIiq0FisRkRAiJydH2NraiiNHjujsf+mll8SYMWO0y0lv3bpV+96dO3eEjY2N2LZtmxBCiLFjx4qnnnpK5/j//Oc/onXr1kIIIaKiogQAsXfvXoMxaK6xb98+7b7du3cLAOLBgwcm+ZxEZDzWYBBRhUVHR+P+/ft46qmnYGdnp902btyI69eva8t169ZN+38nJye0aNECly9fBgBcvnwZPXr00Dlvjx49cO3aNSiVSpw9exZyuRy9e/cuMZb27dtr/+/p6QkAuH37doU/IxGVjYW5AyCimi8rKwsAsHv3bnh7e+u8p1AodJKM8rKxsTGqnKWlpfb/kiQBUPcPIaKqxRoMIqqw1q1bQ6FQIC4uDn5+fjqbj4+PttyxY8e0/09LS8PVq1fRqlUrAECrVq1w+PBhnfMePnwYzZs3h1wuR7t27aBSqXT6dBBR9cUaDCKqsPr162P27NmYNWsWVCoVevbsiYyMDBw+fBj29vZo3LgxAGDx4sVwdnaGu7s75s2bBxcXFwwdOhQA8Oabb6JLly547733MGrUKBw9ehSfffYZVq9eDQDw9fXFhAkTMHnyZHz66afo0KEDbt68idu3b2PkyJHm+uhEVBxzdwIhotpBpVKJ8PBw0aJFC2FpaSlcXV1FcHCw+Ouvv7QdMH/++WfRpk0bYWVlJbp27SrOnTunc44dO3aI1q1bC0tLS9GoUSPx0Ucf6bz/4MEDMWvWLOHp6SmsrKyEn5+f+Prrr4UQjzp5pqWlacv/888/AoCIiYmp7I9PREVIQghh5hyHiGq5AwcOoG/fvkhLS4Ojo6O5wyGiKsA+GERERGRyTDCIiIjI5NhEQkRERCbHGgwiIiIyOSYYREREZHJMMIiIiMjkmGAQERGRyTHBICIiIpNjgkFEREQmxwSDiIiITI4JBhEREZnc/wO2hDMzpvx0SwAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 600x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "\n",
       "<style>\n",
       "    /* background: */\n",
       "    progress::-webkit-progress-bar {background-color: #CDCDCD; width: 100%;}\n",
       "    progress {background-color: #CDCDCD;}\n",
       "\n",
       "    /* value: */\n",
       "    progress::-webkit-progress-value {background-color: #00BFFF  !important;}\n",
       "    progress::-moz-progress-bar {background-color: #00BFFF  !important;}\n",
       "    progress {color: #00BFFF ;}\n",
       "\n",
       "    /* optional */\n",
       "    .progress-bar-interrupted, .progress-bar-interrupted::-webkit-progress-bar {\n",
       "        background: #000000;\n",
       "    }\n",
       "</style>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      <progress value='99' class='progress-bar-interrupted' max='100' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      99.00% [99/100] [02:25<00:01]\n",
       "      <br>\n",
       "      ████████████████████100.00% [2/2] [val_loss=0.1268, val_mae=0.3601]\n",
       "    </div>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;31m<<<<<< val_mae without improvement in 10 epoch,early stopping >>>>>> \n",
      "\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "dfhistory = keras_model.fit(\n",
    "    train_data = dl_train,\n",
    "    val_data= dl_val,\n",
    "    ckpt_path='checkpoint',\n",
    "    epochs=100,\n",
    "    patience=10,\n",
    "    monitor=\"val_mae\", \n",
    "    mode=\"min\",\n",
    "    plot = True,\n",
    "    wandb = False\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "092528f3",
   "metadata": {},
   "source": [
    "## 四，评估模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "15693dba",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "100%|████████████████████████████████| 15/15 [00:00<00:00, 29.20it/s, val_loss=0.196, val_mae=0.296]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'val_loss': 0.19550365308920542, 'val_mae': 0.2955344617366791}"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "keras_model.evaluate(dl_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "4f52f49a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "100%|██████████████████████████████████| 2/2 [00:00<00:00, 15.72it/s, val_loss=0.266, val_mae=0.329]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'val_loss': 0.26627758145332336, 'val_mae': 0.3290509581565857}"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "keras_model.evaluate(dl_val)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "6050f49a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "100%|███████████████████████████████████| 5/5 [00:00<00:00, 25.20it/s, val_loss=0.224, val_mae=0.32]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'val_loss': 0.22361865341663362, 'val_mae': 0.3199106752872467}"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "keras_model.evaluate(dl_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ced7f651",
   "metadata": {},
   "source": [
    "## 五，使用模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "0c3f0dd5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "14ee6f1835734ce1b4c71fe32f868ebb",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/2 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e684c846a1514380b185d08c4a8027c9",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/5 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "val_acc =  0.843709693736275\n",
      "test_acc =  0.8443259525697211\n"
     ]
    }
   ],
   "source": [
    "from tqdm.auto import tqdm \n",
    "\n",
    "dl_val = keras_model.accelerator.prepare(dl_val)\n",
    "dl_test = keras_model.accelerator.prepare(dl_test)\n",
    "net.eval()\n",
    "\n",
    "def predict(net, dl):\n",
    "    net.eval() \n",
    "    preds = []\n",
    "    with torch.no_grad():\n",
    "        for batch in tqdm(dl):\n",
    "            preds.append(net.predict(batch))\n",
    "    yhat_list = [yi for yd in preds for yi in yd.cpu()]\n",
    "    yhat = np.array(yhat_list).reshape(-1,)\n",
    "    return yhat \n",
    "    \n",
    "\n",
    "def accuracy_score(yhat,y):\n",
    "    total_diff = np.abs(yhat - y).sum()\n",
    "    total_gt = y.sum()\n",
    "    return  1.0  - total_diff/total_gt \n",
    "\n",
    "yhat_val = predict(net,dl_val)\n",
    "yhat_test = predict(net,dl_test)\n",
    "\n",
    "\n",
    "val_acc = accuracy_score(yhat_val,dfval[target_col])\n",
    "test_acc = accuracy_score(yhat_test,dftest[target_col])\n",
    "\n",
    "print('val_acc = ', val_acc)\n",
    "print('test_acc = ', test_acc)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "f400aaa8-cde1-44dc-91c2-353e295b19c5",
   "metadata": {},
   "outputs": [],
   "source": [
    "from torch import nn "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8bd458bc",
   "metadata": {},
   "source": [
    "## 六，保存模型"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "84fd68e5",
   "metadata": {},
   "source": [
    "最佳模型权重已经保存在ckpt_path = 'checkpoint'位置了。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "039d1fd4",
   "metadata": {},
   "outputs": [],
   "source": [
    "net.load_state_dict(torch.load('checkpoint',weights_only=True))"
   ]
  }
 ],
 "metadata": {
  "kaggle": {
   "accelerator": "gpu",
   "dataSources": [],
   "dockerImageVersionId": 30747,
   "isGpuEnabled": true,
   "isInternetEnabled": true,
   "language": "python",
   "sourceType": "notebook"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
